perf(query): evaluate CONTAINS, STARTS WITH, ENDS WITH with Arrow kernels by prrao87 · Pull Request #148 · hyperdimensionalcomputingai/lance-csr

prrao87 · 2026-11-10T14:35:11Z

Fixes #146.

Problem

Since #120, CONTAINS, STARTS WITH and ENDS WITH run through the __string_* UDF, which handles bound parameters, %/_/\ and Cypher's null for non-string operands. But it evaluated row by row. It broadcast a literal search string into an array as long as the batch, then called str::contains per row, which builds a new substring searcher each time. On LDBC that made CONTAINS filters up to about 2x slower than 0.7.0's LIKE.

Change

The UDF keeps its place in the plan and its semantics. Its body now calls Arrow's contains / starts_with / ends_with kernels, the ones DataFusion's LIKE and contains() use. A literal or parameter pattern stays a scalar, and one searcher serves the whole batch. A column pattern (a.name CONTAINS b.name) still goes row by row through the same kernels. Plans and explain() output are unchanged.

Tests

Added a column-pattern case to test_string_parameters.py; the existing cases there cover %, _, \, non-ASCII, empty, null and non-string patterns, for both join and CSR plans.
make install && uv run --no-sync pytest (1,714 passed), cargo test -p lance-csr --lib, cargo clippy -p lance-csr --all-targets -- -D warnings.

Benchmarks

Release wheels from main (45841ad) and this branch (d7dfe35), run with the CONTRIBUTING harness, alternating main/branch, two runs each (three for complex/memory, see below). Apple silicon, plugged in. Every run passed its answer checks.

Suite	Mode	Geomean branch/main
graph-benchmark-ldbc	memory	0.962 (−3.8%)
graph-benchmark-ldbc	lazy	0.966 (−3.4%)
graph-benchmark-ldbc-complex	memory	1.039 (+3.9%), see note
graph-benchmark-ldbc-complex	lazy	1.006 (+0.6%)

Eight queries in this suite use CONTAINS. The gains are in three of them: Q5 −25% (memory) / −36% (lazy) and Q20 −40% / −29%, the two queries from the issue, and Q19 −8% / −10%. The other five (Q2, Q6, Q8, Q16, Q29) and the rest of the suite are within noise.

Note on complex/memory: no query in that suite uses CONTAINS, STARTS WITH or ENDS WITH, so this change doesn't reach it. Q7 (+20%) and Q9 (+26%) come from one branch run (0004) with interference. Its Q7 mean was 16.2 ms against a median of 10.0 ms, and Q9's 72.5 ms against 49.5 ms. The other two branch runs match main: Q7 at 9.7 and 10.0 ms against main's 9.8 to 10.1 ms, and Q9 at 42.7 and 44.8 ms against main's 40.9 to 43.6 ms. The table below averages all three runs, outlier included.

The issue's synthetic reproducer (1M rows, medians of 15 runs, two rounds):

1M rows	main	branch
`CONTAINS`, 2 words	1.83–1.90 ms	1.39–1.54 ms
`CONTAINS`, 20 words	2.16 ms	1.71–1.74 ms
`STARTS WITH` / `ENDS WITH`, 2 words	1.30–1.38 ms	0.55–0.63 ms
`STARTS WITH` / `ENDS WITH`, 20 words	2.4–2.8 ms	1.75–1.82 ms

CONTAINS on short strings is back near 0.7.0's 1.4 ms.

Every query (mean of runs; "main run gap" is the difference between the first and last main runs)

graph-benchmark-ldbc, memory (2 main runs, 2 branch runs)

query	main (ms)	branch (ms)	change	main run gap
Q1	0.64	0.61	-4.8%	9.6%
Q10	0.71	0.71	-0.5%	1.8%
Q11	1.53	1.54	+0.3%	0.4%
Q12	0.75	0.75	+0.6%	0.3%
Q13	0.80	0.80	+0.5%	0.7%
Q14	0.52	0.53	+1.6%	0.0%
Q15	0.56	0.56	+1.1%	0.4%
Q16	0.67	0.66	-0.7%	0.1%
Q17	0.90	0.88	-2.4%	1.8%
Q18	0.70	0.70	+0.4%	1.0%
Q19	1.02	0.94	-7.5%	2.1%
Q2	0.55	0.53	-3.5%	1.5%
Q20	3.68	2.21	-40.1%	2.2%
Q21	0.77	0.72	-5.3%	9.7%
Q22	0.71	0.70	-1.6%	1.4%
Q23	0.73	0.74	+0.2%	2.7%
Q24	0.69	0.69	-1.0%	0.5%
Q25	0.79	0.79	+0.3%	1.3%
Q26	0.74	0.73	-0.1%	0.7%
Q27	0.87	0.88	+0.2%	1.9%
Q28	1.43	1.44	+0.6%	1.1%
Q29	0.69	0.69	-0.4%	2.7%
Q3	0.65	0.63	-3.0%	3.8%
Q30	46.11	44.12	-4.3%	1.1%
Q4	0.50	0.50	-0.2%	1.0%
Q5	1.05	0.79	-24.6%	1.0%
Q6	0.46	0.43	-6.0%	10.9%
Q7	1.02	1.02	-0.3%	2.7%
Q8	0.63	0.63	+0.8%	1.6%
Q9	0.81	0.81	+0.5%	0.2%

Geomean ratio branch/main: 0.962 (-3.8%) over 30 queries

graph-benchmark-ldbc, lazy (2 main runs, 2 branch runs)

query	main (ms)	branch (ms)	change	main run gap
Q1	1.18	1.16	-1.1%	1.6%
Q10	0.92	0.92	-0.4%	0.4%
Q11	2.01	1.98	-1.7%	2.7%
Q12	6.91	6.65	-3.7%	10.2%
Q13	1.05	1.05	-0.2%	1.2%
Q14	0.68	0.68	+0.0%	1.3%
Q15	0.76	0.75	-1.9%	3.6%
Q16	1.11	1.11	-0.2%	0.6%
Q17	1.37	1.36	-0.3%	0.7%
Q18	1.14	1.14	+0.4%	0.8%
Q19	1.51	1.36	-10.0%	1.5%
Q2	1.40	1.40	-0.0%	0.2%
Q20	23.53	16.69	-29.1%	1.5%
Q21	0.83	0.83	+0.0%	0.3%
Q22	0.91	0.92	+0.4%	1.7%
Q23	1.25	1.24	-1.3%	0.0%
Q24	1.07	1.06	-0.7%	1.2%
Q25	1.20	1.20	+0.5%	1.1%
Q26	1.09	1.09	+0.6%	2.0%
Q27	1.31	1.32	+0.9%	0.4%
Q28	2.06	2.08	+1.0%	1.0%
Q29	1.03	1.03	+0.5%	1.2%
Q3	0.79	0.78	-1.0%	1.2%
Q30	45.91	46.11	+0.4%	3.8%
Q4	0.79	0.77	-1.8%	2.5%
Q5	3.58	2.28	-36.4%	2.2%
Q6	0.74	0.73	-1.8%	0.3%
Q7	1.08	1.07	-0.6%	0.9%
Q8	1.75	1.74	-0.8%	0.7%
Q9	1.33	1.33	-0.0%	0.2%

Geomean ratio branch/main: 0.966 (-3.4%) over 30 queries

graph-benchmark-ldbc-complex, memory (3 main runs, 3 branch runs)

query	main (ms)	branch (ms)	change	main run gap
Q1	24.04	24.23	+0.8%	4.5%
Q10	20.99	22.25	+6.0%	4.1%
Q11	3.05	3.10	+1.6%	0.5%
Q12	19.60	18.33	-6.5%	7.1%
Q13	1.12	1.11	-1.2%	8.0%
Q14	51.30	50.65	-1.3%	6.8%
Q2	2.87	2.91	+1.4%	0.5%
Q3	26.80	27.62	+3.1%	0.9%
Q4	6.19	6.13	-1.1%	0.3%
Q5	80.44	82.66	+2.8%	1.2%
Q6	25.07	25.59	+2.1%	4.9%
Q7	9.97	11.98	+20.1%	0.0%
Q8	2.30	2.43	+5.5%	1.5%
Q9	42.48	53.32	+25.5%	6.1%

Geomean ratio branch/main: 1.039 (+3.9%) over 14 queries

graph-benchmark-ldbc-complex, lazy (2 main runs, 2 branch runs)

query	main (ms)	branch (ms)	change	main run gap
Q1	32.83	32.27	-1.7%	2.2%
Q10	29.22	29.50	+0.9%	1.2%
Q11	5.58	5.54	-0.8%	2.3%
Q12	20.55	20.91	+1.7%	5.5%
Q13	1.42	1.43	+0.5%	0.2%
Q14	52.04	51.67	-0.7%	0.4%
Q2	25.91	26.87	+3.7%	0.2%
Q3	167.13	167.12	-0.0%	0.4%
Q4	13.36	13.29	-0.5%	6.0%
Q5	102.12	106.59	+4.4%	4.9%
Q6	26.77	26.02	-2.8%	1.6%
Q7	13.87	14.03	+1.1%	3.5%
Q8	3.35	3.36	+0.1%	0.3%
Q9	314.73	321.74	+2.2%	2.2%

Geomean ratio branch/main: 1.006 (+0.6%) over 14 queries

🤖 Generated with Claude Code

…nels The string predicate UDF broadcast a literal search string to a full array and called str::contains per row, building a new substring searcher each time. Call Arrow's contains/starts_with/ends_with kernels instead: a literal stays a scalar and one searcher serves the batch. These are the kernels DataFusion's LIKE and contains() use. Bound parameters, null and non-string operands keep their Cypher semantics. Fixes #146

cloudflare-workers-and-pages · 2026-11-10T14:35:35Z

Deploying lance-csr with Cloudflare Pages

Latest commit:	`d7dfe35`
Status:	✅ Deploy successful!
Preview URL:	https://d988ea81.lance-csr.pages.dev
Branch Preview URL:	https://perf-native-string-predicate.lance-csr.pages.dev

View logs

A patch bump: #147 fixes an explain() panic and #148 speeds up string predicates, with no API changes.

prrao87 mentioned this pull request Nov 10, 2026

chore: bump version 0.9.0 → 0.9.1 #149

Merged

prrao87 merged commit c6a9abd into main Nov 10, 2026
7 checks passed

prrao87 deleted the perf/native-string-predicates branch November 10, 2026 14:38

prrao87 added a commit that referenced this pull request Nov 10, 2026

chore: bump version 0.9.0 → 0.9.1 (#149)

c062ac2

A patch bump: #147 fixes an explain() panic and #148 speeds up string predicates, with no API changes.

Navigation Menu

Repository navigation

perf(query): evaluate CONTAINS, STARTS WITH, ENDS WITH with Arrow kernels - #148

perf(query): evaluate CONTAINS, STARTS WITH, ENDS WITH with Arrow kernels#148
prrao87 merged 1 commit into
mainfrom
perf/native-string-predicates

prrao87 commented Nov 10, 2026 •

edited

Loading

Uh oh!

cloudflare-workers-and-pages Bot commented Nov 10, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

prrao87 commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Change

Tests

Benchmarks

graph-benchmark-ldbc, memory (2 main runs, 2 branch runs)

graph-benchmark-ldbc, lazy (2 main runs, 2 branch runs)

graph-benchmark-ldbc-complex, memory (3 main runs, 3 branch runs)

graph-benchmark-ldbc-complex, lazy (2 main runs, 2 branch runs)

Uh oh!

cloudflare-workers-and-pages Bot commented Nov 10, 2026

Deploying lance-csr with Cloudflare Pages

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

prrao87 commented Nov 10, 2026 •

edited

Loading